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Mobile telephone-operated video telesurveillance device, 
operating method, application and telesurveillance 

network 

5 The present invention concerns a mobile telephone- 

operated video telesurveillance device as well as an operating 
method, applications as well as a public telesurveillance 
network. It enables remote surveillance with a stand-alone 
decision-making process in relation to a video environment 

10 captured by the mobile telephone. It may be implemented in 
professional fields, for instance safety of premises, road 
conditions and in particular climate or traffic conditions 
(speed of vehicles, jumping traffic lights), as well as in private 
fields, for instance surveillance of a toddler, surveillance of a 

15 swimming-pool or other systems in a residence. 

As technologies evolve, the video surveillance is taking 
over in more and numerous fields. The multiplicity of the 
devices for each application brings about more and more 
sophisticated network mechanism. 

20 A video surveillance device contains a camera, an 

analysis and diagnostic means of video pictures based on the 
application, and a means for communicating the alarms and 
programming the device. 

In parallel, the telephone becomes more and more 

25 perfected, after having fulfilled for a long time its primary 
function as a remote sound link, it has become mobile and 
integrates today visual communication, shot by a camera and 
visualised on a flat screen. 

The telephone operators are seeking continuously for 

30 new uses of the mobile telephone and the latter becomes a 
video game companion which may be disturbed occasionally 
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by an SMS and sound communication has now been placed in 
the background. 

It is thus that in the application EPO 884 905 concerning 
a production process of a picture to be transmitted over a 
5 terminal, it has been suggested using a telephone with a 
camera comprising a shape recognition means enabling to 
localise the user's face for transmission. In the application 
EP1 170 953 concerning a mobile telephone, remote 
surveillance system, mobile information terminal and method 

10 of use, remote surveillance has been suggested, by 
transmitting pictures in relation to a transmission 
authorisation signal preventing third parties from accessing 
the picture. Such methods and devices are however under the 
user's control for the operation thereof and the transmission 

15 of pictures. 

The purpose of the present invention is to create a video 
surveillance device from a telephone possessing visual 
communication means by the integration of a picture analysis 
module having its own diagnostic and dialogue processing 

20 unit with, at the input to the module, reading the video signal 
issued from the camera and, at output, a link to the control 
member of the telephone in order to enable activation/ 
deactivation and stand-alone operation of said telephone. 

Besides, methods and devices are known, which enable 

25 to locate objects in a picture. Some of them suggest statistic 
analysis of the points or pixels of a digital video signal issued 
from an observation system for the realisation of efficient 
devices liable to operate in real-time. 

More recently, it has been suggested to realise such 

30 devices by the association of information processing units of 
similar nature each addressing a particular parameter 
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extracted from the video signal for the analysis thereof. It is 
the case of the patent applications of Monsieur Patrick PIRIM, 
inventor: FR96/09420 (tracking a moving zone), FR96/09420 
and FR98/00378 (monitoring a driver's vigilance), FR98/16679 
5 and WO99/00425 (compression system), WO98/05383 
(picture processing), FR00/02355 (automatic perception), 
FR01/02539 filed on 23 February 2001 (tracking an object), 
FR01/02530 (adaptation), FR01/10750 (processing system), 
FR02/10067 (visual perception), FR02/10066 (analysis), 

10 FR02/10065 (functional unit), FR02/10064 (dynamic 
recruiting) as well as their extensions whereof in particular 
WO98/05002, WO98/05383, W099/36894, W099/36893, 
WO00/11610, WO00/11609, WO00/13420, WO01/63557 or 
the article "The vision mechanism is integrated on a single 

15 chip" in Electronique, June 2000 N°104 wherein the 
implementation of histogram calculation and processing 
blocks or units or modules, such terms being here equivalent, 
is suggested, forming true electronic Space-Time-division 
Neurons, so-called STN, each analysing a parameter, said 

20 parameter being processed by a function (f Q g) to generate 
individually an output value. Such output values, altogether, 
form a feedback available on a bus for usage in the blocks 
during the analysis. At the same time, each of these 
histogram calculation and processing units constitutes and 

25 updates an analysis output register providing with statistic 
information on the corresponding parameter. The selection of 
the parameter analysed by each histogram calculation and 
processing block, the content of the analysis output register 
as well as the function (f Q g) that it fulfils, are determined by a 

30 software executed in an API (Application Program Interface). 



4 



There is explained that each STN unit may fulfil 
automatic classification, anticipation and learning functions. 

In a histogram calculation and processing unit as 
described in WO01/63557, for a given parameter, several 
5 elements are determined from the histogram calculated and 
stored in a memory, i.e. the maximum RMAX of the histogram, 
the position of said maximum POSRMAX, a number of points 
NBPTS of the histogram. Also classification milestones are 
also determined, which enable to delineate an interest zone 

10 for the parameter and it has been suggested to consider as a 
criterion for the determination of the milestones, a ratio of the 
maximum of the histogram, for instance RMAX/2, and to 
obtain the milestones by scanning the data of the memory 
since the origin, searching for zone limits corresponding to 

15 the criterion. 

An application of the STN blocks is more particularly 
detailed in the application FR-01/02539 where it has been 
suggested to break down hierarchically the object to be 
tracked according to its properties which enables, for 

20 instance, to determine first of all the overall contour of a 
moving object relative to relatively stable background, then 
look inside said contour for characteristic elements by their 
tone, their colour, their relative position..'. Such an analysis 
enables rapid elaboration of multiple applications involving 

25 tracking an object. Such applications may be developed, 
either from a prior formalisation having outlined the significant 
characteristics of the object, or, thanks to a learning function 
by examining a scene wherein the object in question is 
present, the device itself enabling to extract characteristic 

30 parameters of the object. Below, the term element or object 
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will be used interchangeably regarding what should be tracked 
in a scene of the environment. 

The invention suggesting integrating to a mobile 
telephone possessing sensors of an environment, in particular 
5 of a sound and picture environment, a module enabling 
analysis of the signals issued from the sensors and to conduct 
a diagnostic before making a decision. This decision-making 
process may be an absence of action in particular in case of 
negative diagnostic or one or several actions in case of 

10 positive diagnostic (for example recognising and exceeding a 
parameter threshold of an element of a scene). The analysis 
consists in extracting and measuring element parameters 
significant of the environment. For instance, if the analysis 
must bear upon a face: recognising the face within the scene 

15 then, in the face, recognising the eyes, the mouth then, 
possibly, within these elements of the face, determine 
significant sub-elements: eyelid, pupil or lips... The (diagnostic 
consists in determining or detecting the existence of particular 
elements of a scene and/or exceeded parameter thresholds 

20 linked with an element in the scene. Within the context of the 
invention, the term scene covers anything which is captured 
by a sensor, for a camera, it is a series of pictures, for a 
microphone, sounds, for a temperature sensor a temperature 
measurement... and the corresponding element may for 

25 instance be a moving object/human in the pictures, sounds of 
a particular frequency range and/or a duration and/or a sound 
intensity, temperature as such. 

The invention concerns therefore a mobile telephone- 
operated video telesurveillance device, the telephone 

30 comprising a transmitter and a receiver operated by a control 
member for connection to a telephone network, the telephone 
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comprising means for capturing sound signals BFm and video 
signals Vc by dint of a microphone outputting the sound 
signals BFm and of a camera outputting the video signals Vc 
of video pictures and, on the other hand, receiving and 
5 reproducing sound signals BFe and video signals Ve intended 
for, respectively, a sound reproduction means and of a video 
reproduction screen, the telephone comprising a user input 
interface and a stand-alone power supply source. 

According to the invention, the telephone includes 

10 moreover an analysis and diagnostic module of at least video 
pictures fitted with a memory for storing at least one 
telesurveillance application program, an input to said module 
being connected to the output of the camera and receiving the 
video signals Vc, the module being connected by at least one 

15 output with the control member for sending at least one 
diagnostic datum D to said control member in relation to the 
analysis and to the diagnostic of the video signal by a 
calculation means programmed by an application program of 
said module, the diagnostic data D being instructions acting 

20 upon the control member. 

Within the framework of the invention, the term video for 
the analysis module concerns fixed or animated pictures as 
well as sounds and concerns more generally which is called 
commonly multimedia. 

25 In various embodiments of the invention, the following 

means, taken individually or in combination, are employed: 

- the camera is a camcorder, 

- the camera is of the CCD-type (load transfer) or CMOS, 

- the program memory of the analysis and diagnostic module 
30 is preloaded by the application program (the program is 

available at the onset) 
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- the application program is downloaded in the program 
memory of the analysis and diagnostic module, 

- the application program includes at least one call number on 
the network, 

5 - the application program includes at least one computer call 
address on the network, 

- the analysis and diagnostic module receives moreover at 
input the sound signals BFm and the module generates 
diagnostic data D also in relation to the analysis and to the 

10 diagnostic of the sound signals, 

- the instructions acting upon the control member are one or 
several actions selected by no way of limitation among the 
following list: 
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waking up telephone members in standby, 
placing telephone members in standby, 
calling a number over the network, 
transmitting recorded sounds, 
live transmission of sounds, 

transmission of a short message (in particular SMS), 
transmission of a live video sequence, 
transmission of a recorded video sequence, 
transmission of a video analysis, 
transmission of a video diagnostic, 

transmission of multimedia compatible data for 
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INTERNET, 



dialogue between the telephone and the number called, 
in case of using a positioning peripheral, controlling said 
peripheral via positioning instructions from the network, 
end of call, 



the analysis and diagnostic module includes a 
reprogrammable memory for the program of the calculation 
means, 

- the analysis and diagnostic module receives moreover digital 
5 data D from the control member and the program of the 

calculation means comes from the network and is transferred 
into the reprogrammable memory in the form of digital data D, 

- the analysis and diagnostic module receives moreover digital 
data D from the control member, the telephone comprising 

10 moreover a chip card reader legible by the control member 
and the program of the calculation means comes from the 
chip card and is transferred into the reprogrammable memory 
in the form of digital data D, 

- the analysis and diagnostic module includes a video picture 
15 acquisition and memorisation module and a sound detection 

means for interfacing with the video and sound signals, a 
digital signal processor or micro-processor-type calculation 
means, the memory for the application program and a 
dialogue unit for interfacing with the control member, 
20 - the analysis and diagnostic module includes at least one 
spatial and time-division processing unit and a set of 
histogram calculation units whereof the material and 
functional configuration depends on the application program, 

- the analysis and diagnostic module is a block IP which may 
25 be integrated (materially and/or functionally) into an electronic 

circuit, 

- the electronic circuit is an electronic component, 

- the electronic component is an integrated circuit comprising 
one/several telephone functions (component with one/several 

30 usual functions in telephony: for instance transmission, 
reception, signal processing, signalling...) of the telephone, 
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- the telephone is arranged on a positioning peripheral 
enabling at least to direct said telephone in space according 
to positioning instructions (OM) issued from the telephone, 
said positioning instructions being generated by the analysis 

5 and diagnostic module, 

- the positioning peripheral includes a first bracket wherein 
the telephone is arranged, the first bracket being itself 
arranged in a second bracket, the first and the second bracket 
being articulated together by at least one motor operated by a 

10 control circuit receiving the positioning instructions (OM), 

- the second bracket includes two motors whereof both rotors 
are laid out along two axes substantially perpendicular to one 
another, the control circuit and a charger intended for 
recharging the stand-alone power supply source of the 

15 telephone, 

- the positioning peripheral of said telephone enables 
moreover to move in space said telephone according to 
positioning instructions (OM), 

- moreover, positioning instructions transit over the network, 
20 - the positioning instructions (OM) are transmitted to the 

positioning peripheral device by dint of a telephone connector 
comprising a series data transmission link, 

- the telephone includes one or several of the following 
sensors whereof the output is connected to the analysis and 

25 diagnostic module, said module analysing and diagnosing the 
measurements of said sensors: 

- angular position sensor, 

- acceleration sensor, 

- speed sensor, 

30 - temperature sensor, 

- luminosity sensor, 
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- barometric pressure sensor, (possible use as a burglar 
alarm: the opening of a door causing a pressure wave in 
premises), 

- biometric sensor (fingerprints, ADN...), 

- the sensors of the telephone are connected to the telephone 
by dint of a telephone connector comprising a series data 
transmission link. 

The invention also concerns therefore a positioning 
peripheral compatible with the telephone of the invention and 
enabling at least to direct said telephone in space according 
to positioning instructions (OM) issued from the telephone, 
said positioning instructions being generated by the analysis 
and diagnostic module. 

The invention also concerns an operating method which 
derives from the implementation of the device as described 
previously according to all its possibilities and which is an 
operating method of a mobile telephone-operated video 
telesurveillance device, the telephone comprising a 
transmitter and a receiver operated by a control member for 
connection to a telephone network, whereas the telephone 
can, on the one hand, capture and transmit sound signals 
BFm and video signals Vc issued respectively from a 
microphone generating the sound signals BFm and from a 
camera generating the video signals Vc of video pictures and, 
on the other hand, receiving and reproducing sound signals 
BFe and video signals Ve intended for, respectively, a sound 
reproduction means and a video reproduction screen, the 
telephone comprising a user input interface and a stand-alone 
power supply source, wherein pictures issued from the 
camera are transmitted to a receiving apparatus connected to 
the network and which can visualise the video pictures, 
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wherein a device is implemented according to any of the 
previous features individually or in combination and wherein 
the telephone includes moreover an analysis and diagnostic 
module of at least video pictures, fitted with a memory for the 
5 storage of a telesurveillance application program, an input to 
said module being connected to the output of the camera and 
receiving the video signals Vc, the module being connected by 
at least one output with the control member for sending at 
least one diagnostic datum D to said control member in 

10 relation to a decision resulting from the analysis and the 
diagnostic of the video signal by a calculation means 
programmed by an application program of said module, the 
diagnostic data D being instructions acting upon the control 
member and in that the connection to the receiving apparatus 

15 results from the decision generated by the analysis and 
diagnostic module. 

According to variations of the method, the receiver is a 
mobile or fixed telephone and/or the receiver is a 
computerised means capable of visualising multimedia 

20 compatible data for INTERNET, a server connected over the 
network and arranged between the telephone and the receiver 
providing the interface between the telephone data and the 
INTERNET network. 

The invention also concerns a particular application of 

25 the previous operating method wherein a toddler is monitored 
remotely, the application program enabling at least the 
analysis and the diagnostic of the toddler's movements and 
the connection to the receiver when movements are 
diagnosed. 

30 The invention concerns finally a public telesurveillance 

network which is composed of a mobile phone network 
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comprising a transmission exchange, over such network, 
application telesurveillance programs for a telephone- 
operated video telesurveillance device, programmes which 
may be downloaded by said telephones. 
5 The mobile telephone of the invention includes therefore 

stand-alone decision-making means in relation to the analysis 
and to the diagnostic of an environmental context (pictures, 
sounds or others) captured by the sensors of the telephone 
and may therefore of its own initiative decide to alert 

10 compatible remote equipment (conventional or specialised 
telephone, for instance the equipment used by the firemen, 
the police and the surveillance exchanges or computerised 
means): the decision-making process unfurls automatically. 
Moreover, the analysis and the diagnostic being dependant on 

15 an application program, the analysis and diagnostic 
parameters may be suited to the application contemplated 
and, preferably, by reprogrammable means and in particular 
by transmission over the telephone network of the program to 
the telephone of the invention or by inserting in the telephone 

20 a chip card containing the program which is then transferred 
to the analysis and diagnostic module of the telephone. 

This decision-making process unfurling upstream, at the 
source of the environment, the downstream user is relieved 
from the surveillance tasks and, moreover, the network is 

25 freed, the connection and transmission being initiated only 
according to the result of a diagnostic. 

The use of this method renders the video surveillance 
market most casual, and the notion of associated services 
becomes predominant. On top of the fact that the mobile 

30 telephone only uses the network for the transmission of digital 
data, diagnostic data (alarm and type of alarm for instance) 
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and/or video data in relation to the analysis and to the 
diagnostic of a video environment, typically preloaded or 
downloaded application programs, from the downloading 
transmission exchange, which reduces the weight of the 
5 telesurveillance on the network as regards the pass-band 
volume, fine analysis of the picture by elaborated visual 
perception may enable moreover to compress the video 
information to be transmitted. 

It should be noted finally that, within the framework of 

10 the invention, the mobile telephone may be intended for a 
public network as well as a private network, for instance 
within a company or a fixed or mobile building. Besides, the 
implementation of a mobile telephone is considered preferably 
since the former, further to its development, is now relatively 

15 cheap to produce and its modification for adaptation of an 
analysis and diagnostic module should only increase the 
production cost marginally. However, the invention is more 
generally applicable to means for radiocommunications of 
sounds, video (pictures) and network data developed 

20 specifically to that end. 

The present invention will now be exemplified in relation 

to: 

Figure 1 which represents a video mobile telephone of the 
state of the art, 

25 Figure 2 which represents a mobile telephone according to the 
invention with an analysis and diagnostic module, 
Figure 3 which represents an analysis and diagnostic module 
realised with conventional calculation means, 
Figure 4 which represents an analysis and diagnostic module 

30 realised with space-time-division neuron-type calculations 
means, 
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Figure 5 which represents an application example of the 
telephone of the invention, 

Figure 6 which represents the schematic material structure of 
a second application example of the telephone of the 
5 invention, and 

Figure 7 which represents a telephone in this second 
application example of the invention. 

Figure 1 represents a simplified flow chart of a mobile 
telephone 1 integrating visual communication, corresponding 

10 to the previous art. This telephone includes a basic core with 
a transmitter 2 and a receiver 4 operated by a control member 
8. Moreover, to enable an interaction with the user, control 
interface means 9 at input with a keyboard and/or voice 
recognition are connected by an electronic bus Tc to the 

15 control member 8. On the basic core are connected at input, a 
microphone 3 sending a low-frequency voice signal BFm and 
a video signal Vc issued from a camera 6 of the telephone 1. 
On the basic core are also connected at output, a headset 5 
or a loudspeaker for sound diffusion of a signal BFe issued 

20 from the receiver 4, as well as a video screen 7 which may 
reproduce video data Ve received, stored in the telephone or 
captured by the camera 6 of the telephone as well. In addition 
to the reproduction role of the video screen 7, said screen, as 
control interface means at output, may also enable interaction 

25 with the user by displaying menus or others. It is also possible 
in certain versions that the sound reproduction means, in 
particular headset 5, form a control interface means at output 
by diffusing instructions or sound menus. The telephone 1 
being mobile, a stand-alone power supply source is integrated 

30 to said telephone. The functional representation of the mobile 
telephone on Figure 1 being very schematised by reason of 
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simplification, all the elements of the telephone have not been 
represented in detail, such as the hardware safety means 
(chip card) and software safety means (encoding/decoding 
programs) or network management and communication means 
5 (network searching programs, network connection 
programs...). 

Figure 2 enables to visualise how the invention applies 
extremely simply to a state-of-the-art telephone. Indeed, the 
invention consists roughly in incorporating in a mobile 

10 telephone of the type which has been described in relation to 
Figure 1, an analysis and diagnostic module video 20. This 
module receives at input at least the video signal Vc issued 
from the camera 6 of the mobile telephone 1. In an enhanced 
version, the module 20 may receive moreover the voice signal 

15 BFm issued from the microphone 3. At output the module 20 
is connected by a digital link D to the control member 8 in 
order to transfer diagnostic data thereto. It will be observed at 
later stage that the module 20 may also take into account 
certain functions from the control member and, for instance, 

20 the compression of data, in particular video data. It may be 
noted that the link D is indicated as bi-directional and it will 
be seen that the module 20 may also receive digital data 
issued from the control member and, for instance 
programming data, in particular the application program, for 

25 the analysis and the diagnostic issued in particular from the 
network and received by the mobile telephone 1. Indeed, the 
invention may be implemented within a network structure 
wherein a supervisor/controller operator provides means of 
communication (among which gates between computer-based 

30 and telephone networks) and of management (in particular 
transmission of application programs enabling the analysis 
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and the video/audio diagnostic in the module of the 
telephone, the programs of the module being downloaded) 
between users of the telesurveillance device. 

In other embodiments, the analysis and diagnostic 
5 module is an additional entity protected in particular by the 
copyright and/or by semi-conductor topography and/or patent 
rights and/or other sui generis rights, which is qualified as a 
block IP (or virtual component which represents knowledge 
linked with the design of functions which may be incorporated 

10 into integrated circuits), and which is integrated within a 
component with conventional telephone functions: the entity is 
added in a component or circuit used conventionally in 
telephony (functionally by adding a program and/or materially 
by adding a functional hardware block of logic gates and/or 

15 mixed by configuration or reconfiguration of a programmable 
structure of logic gates or combination of program and 
hardware). The blocks IP are functional entities now widely 
used for the realisation of integrated circuits and they 
correspond to hardware objects (gate structures for instance) 

20 or software objects (instructions) or a combination of both. 
The blocks IP enable the creation of function libraries wherein 
the designers of integrated circuits may dig for the realisation 
of specific circuits. With reference to the previous example, 
the set of blocks IP intended for the telephony is added a 

25 block IP corresponding to the analysis and diagnostic module 
to realise an integrated circuit intended for the realisation of a 
telephone according to the invention. 

Preferably, the analysis and diagnostic module is a 
single integrated circuit, possibly associated with external 

30 memory, which is arranged in a telephone with conventional 
video means. The addition of such an integrated circuit may 
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take place very simply with minimum modification of the 
extant circuits of the telephone. In particular, the link of the 
module to the control member may be made by connection to 
a communication bus from the control member which is 
5 generally implemented within mobile telephones to control 
peripheral members (transmitter, receiver, keyboard, 
screen...). 

Figure 3 represents the organisation of the picture 
analysis and sound detection module 20 according to a first 

10 embodiment using computerised calculation means, 
conventional calculator, in particular a digital signal processor 
(DSP) or a micro-processor. The video signal Vc derived from 
the camera 6 enters the video picture acquisition and 
memorisation sub-assembly 21. The sound electric signal 

15 BFm derived from the microphone 3 enters the sound 
detection sub-assembly 22. A calculator 23 runs the 
application program(s) 25 which enables (enable) to analyse 
and to perform a diagnostic on the picture and the sound for 
time-division duration, i.e. a preset time range. This (these) 

20 application program(s) may have already been stored in the 
telephone and/or preloaded and/or downloaded from the 
network. In relation to the diagnostic resulting from the 
analysis, and in particular in case of non-conformity with 
diagnostic criteria, a digital signal D compatible with the 

25 control member 8 is generated by a dialogue unit 24. This 
digital signal D, which corresponds to actions in particular on 
the network connection means of the telephone, may be the 
transmission with a preset number on the network of an 
appropriate message (SMS) with visualisation of the scene 

30 recorded and/or live. 
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The means implemented in this embodiment of Figure 3 
are particularly greedy of electric energy and, preferably, the 
analysis and diagnostic means are used in relation to the 
Figure 4 with space-time-division neurons (STN) which are 
5 particularly adapted to these functions and particularly flexible 
to use (re-organisation and re-use of the neurons in relation 
to the depth of analysis and/or of the application), and this 
has relatively reduced energy cost. Said means may also be 
used for purposed of video compression as seen in the 
10 presentation of the state of the art in the first part of the 
description. 

Figure 4 represents a preferred embodiment with space- 
time-division neurons (STN) including by way of example a set 
of sixteen polyvalent (or space-time-division neurons, these 

15 terms being equivalent) histogram units or modules in the 
form of a generic visual perception processor (GVPP) 20' 
which forms the analysis and diagnostic module with space- 
time-division neurons. This processor includes moreover a 
sound detection means 22 acting directly on an electronic 

20 processor 23 itself in relation with a dialogue interface or unit 
24 by a bi-directional bus D to and from the control member 8. 

These histogram calculation units form a matrix, they 
are connected to a bus 56 whereon the parameters DMVT 
(direction of the movement), VMVT (speed of the movement), 

25 L (light), T (tone), S (saturation), pO, p1, p2,..., p15 are 
available (pO, p1, p2,..., p15 are slopes of reference axes 
produced by the unit 54). The bus 57 carries the feedback 
information. Such a processor is in particular described in the 
following publications FR-2. 61 1.063 and WO-98/05002. 

30 This perception processor 20' implements a spatial and time-division 
processing unit 53 which, receiving a video-type signal, produces a 
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certain number of parameters for each pixel. They may be for instance 
the speed (VMVT), the direction (DMVT), a time constant and a binary 
validation parameter on top of a delayed video signal and the different 
frame, line and pixel synchronisation signals. In such a device, the 
5 interest of constituting histograms of these parameters enabling the 
constitution, the manipulation and the exploitation of statistical 
information have been emphasised. The purpose of this type of picture 
processing device is to provide at output a signal which carries for each 
pixel a piece of information significant of the result of the application of 

10 recognition or selection criteria according to classification. These criteria 
are predefined or elaborated by the picture processing methods and 
devices properly speaking. In this same document WO-98/05002, a set 
of calculation modules STN enabling to calculate histograms is described 
in relation to the spatial and time-division processing unit 53. Each 

15 module STN 51 a oo 51 a 33 receives data over a data bus 56 and is 
interconnected to the others by a feedback bus 57 transporting results 
after classification. An application programming interface software API 
enables to set up, relative to one another, the operating modules and 
parameters of the perception processor by dint of a material and 

20 functional configuration bus 55. This software API, as well as the 
application program 25, is managed by the electronic processor 23 which 
is in relation with the set of the modules STN by the bus 55 which also 
returns the calculation results of the modules STN. The whole processor 
GVPP is therefore controlled by the electronic processor or control unit 

25 23, of the micro-controller type, which determines in relation to an 
application program 25 those of the parameters DMVT, VMVT, L, T, S, 
pO, p1, p2,..., p15 which are processed at a given time by a unit STN or 
a dedicated group of polyvalent histogram units and according to 
initialisation (INIT), writing (WRITE) and end (END) signals produced by 

30 a sequencer 52. 
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Each histogram calculation unit is connected to a data bus 56 
which supplies the different parameters to be processed, and a feedback 
bus 57 which provides the classification signal of each neuron STN and 
the learning function signals to the different other neurone STN. Each 
5 histogram calculation unit includes a memory, a classifier and a feedback 
unit. Each unit 1a is liable to have automatic anticipation and learning 
classification functions. 

The generic visual perception processor (GVPP) 520 thus formed 
may be integrated on a single semi-conducting substrate. The number of 

10 polyvalent histogram calculation units depends on the application and the 
manufacturing technologies of semi-conducting components available. 

It has been seen that it was possible to implement the invention 
either using essentially software means operating in generic calculation 
circuits such as a central processing unit or a digital signal processor 

15 (DSP) as that used in relation to Figure 3, or using essentially hardware 
means with one/several dedicated circuits in wired logic or in adaptable/ 
programmable logic as the circuit GVPP shown in relation to Figure 4 
with its STN modules. It can be understood that such a distinction of 
means is not absolute since software means need hardware means to 

20 operate and that the adaptation/programming of the dedicated circuits 
may resort to the software. Thus, even in the case of the dedicated 
circuit of Figure 4, an electronic processor 23 has been implemented 
which enables to supervise the structural and functional operation of said 
dedicated circuit. Moreover, these techniques may be combined, 

25 whereas a portion of the analysis process might involve a dedicated 
circuit and another a generic central unit. The means implemented by the 
invention are therefore both hardware and software, since the latter term 
may be taken in the sense of program but also structuring instructions of 
hardware means as in the case of configuration instructions of logic 

30 circuits with programmable operational structure (networks of 
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programmable logic gates, in particular VHDL - high density logic gate 
circuits - within ASIC). 

In these diverse possibilities of realisation, the means 
implemented enable continuous sequencing (real-time) of the pictures of 
5 a scene issued from the sensor (camera, CCD, CMOS) with sub- 
sequencing of each of the pictures received, this temporally and spatially. 
Each pixel (point of the picture) is known by its value P relative to the 
luminance and/or chrominance and its instant position (i, j, t) where i and 
j define the spatial position within the picture matrix and t the sequence 

10 or the time. The consecutive values of P are processed in the time- 
division field (in a time-division processing unit in the case of a dedicated 
circuit GVPP) for extracting parameters linked with each of the pixels, 
said parameters being selected among the luminance, the tone, the 
saturation, the direction of the movement, the speed of the movement, 

15 the orientation of edges, the curvature of edges, etc. and the respective 
positions i,j of the sequence t are processed spatially (in a spatial 
processing unit in the case of a dedicated circuit GVPP) in order to 
determine spatial orientations. The information thus obtained will be 
analysed statistically and hierarchically in order to perceive the elements 

20 forming the pictures and to reach a diagnostic. Thus, in the case of a 
dedicated circuit GVPP, a dynamic recruiting process of the modules 
STN will be implemented during the various sequences to enable active 
perception of the elements of the picture, the recruitment taking place 
relative to given objectives and in particular whether the type of element 

25 liable to be in the picture is known beforehand (a face) or not (generic 
perceptive analysis). On the basis of the perception, a diagnostic process 
is implemented before making action-oriented decisions, in particular on 
the control member, such as, for instance, the activation of the telephone 
and the envoi of a short message (SMS) by the telephone when an 

30 element perceived and diagnosed as a face appears in the picture. 
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Figure 5 represents an example application of video 
surveillance of a sleeping toddler 103 with parents who make 
a point of being alerted remotely when said toddler wakes up. 
Any movement 101 of the baby and/or sound captured by the 
5 camera and/or the microphone of the telephone 110 are 
analysed and a diagnostic is carried out thanks to the 
application program, by the mobile telephone 110 of the 
invention which has been arranged so that it perceives the 
scene 102 by its camera and its microphone, and transmits 

10 121 an alarm and the scene by dint reception means 122 and 
123 of the telephone network 120 to a second mobile 
telephone 130 of the parents and corresponding to the call 
number requested by the diagnostic portion of the application. 
The second telephone 130 which may be a conventional 

15 mobile telephone, enables to visualise the scene by a screen 
131 and to listen to the sounds by a sound reproduction 
means 132. The second telephone 130 may be a fixed 
telephone with or without video means and, in the latter case, 
the alarm and the sound of the baby only will be received. The 

20 second telephone 130 may be replaced with micro-computer- 
type means with wire or radio link or, even a mobile 
telephone-computer, and in particular by dint of the computer 
network INTERNET which enables the transmission of 
multimedia data including pictures and sounds. In the latter 

25 case, an INTERNET server will interface within the network 
120 between the telephone of the invention 110 and the 
computer means replacing the conventional mobile telephone 
130. 

More generally, the telephone of the invention may also 
30 be applied to the surveillance of a flat or of a building, of a 
vehicle for instance in the case of fire, theft or other, of a 
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swimming-pool within the framework of drowning prevention 
thanks to sound detection (a body falling in the water, rippling 
water) and pictures : mass moving or not at the surface or 
depth of the liquid. 
5 The telephone of the invention may also find an 

application in teleconference and, for instance, be centred 
automatically round a speaker whereof the lips are moving, 
the GVPP analysing the faces and portions of face (eyes, 
mouth...) and the diagnostic being programmed for detecting 

10 the movement of the portion of face corresponding to the lips 
according to an anthropomorphic model. 

By changing the application program, it is possible to 
modify the video surveillance function, for instance in a 
vehicle, recognise the owner or the people entitled to drive 

15 said vehicle, the alarm is transmitted to an addressee which 
may be a safety authority in case of non-recognition and it is 
then recorded and instructions are given to the vehicle 
accordingly. Surveillance applications of the automotive traffic 
on roads or motorways are also possible thanks to the means 

20 for determining the speed of the perception elements enabled 
by the GVPP : fluidity measurement and automatic alarm or 
detecting when the authorised speed has been exceeded and 
transmitting an alarm to means for repressing road violence. 

In relation to the application, the receiver station of the 

25 alarm (130) may for instance also be a enhanced telephone 
such as firemen or police or surveillance exchange. Thus, the 
receiving apparatus may be a specific device. 

Other applications are possible as will now be seen in 
relation with Figure 6 where the telephone of the invention 

30 enables to control, in relation to the analysis and to the video 
and/or audio diagnostic, a positioning peripheral device 
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instead of said telephone. The position of the telephone may 
concern the orientation of the shot (whereas the telephone 
may rotate around itself to direct the camera in a given 
direction), the telephone then remaining in a given location. 
5 The position may also concern in combination or non with the 
orientation, the selection of the location in space of the 
telephone, whereas the latter may move in said space. These 
possibilities depend on the positioning peripheral 
implemented, from a single rotary bracket which may be 

10 controlled according to one or two axes, a mobile device 
travelling on a surface (controlled motorised cart), let alone in 
a three-dimensional space (airship, aircraft or sub-marine 
controlled as a reduced-scale model). It can be understood 
that the positioning notion is relative and that if it is generally 

15 the telephone of the invention which is positioned in a fixed 
space (is directed or moved in space for obtaining a direction 
and/or origin with a different viewing point), the case is 
contemplated when the space moves and the aim is to keep a 
direction and/or origin with constant viewing point. The latter 

20 case may, for instance, correspond to an automated beacon 
with video telephone located at the front of a ship liable to be 
roll or pitch and intended to prevent any collision by video 
detection of obstacles : the analysis and diagnostic module is 
then configured to detect floating bodies ahead of the ship 

25 towards the horizon. 

The adaptation of the telephone with video means of the 
invention to such an application is relatively simple since in 
most cases, the mobile telephone possesses a connector 
intended for power recharging and data exchanges (generally 

30 by a series link) between the control member and external 
peripherals. Figure 6 represents under the general reference 
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1 a telephone as that of Figure 2 with a linking connector 101 
for recharging the power supply 10 by the line V and for data 
exchange communication by the link OM represented here as 
unidirectional exchange from the control member outwardly 
5 but which enables more generally bi-directional data 
exchanges. 

The telephone 1 is positioned stably in a first bracket 
100. This first bracket 100 is itself arranged in a second 
bracket 106 in a steerable fashion angularly by control 

10 positioning means, the second bracket being assumed as 
arranged on a fixed location of the space in this version. The 
telephone 1 is hence fixed with respect to the first bracket 
100 and the first bracket 100 may be oriented angularly with 
respect to the second bracket 106. Consequently, the 

15 telephone and hence the shooting direction of a picture may 
be oriented angularly in space. The bracket system presented 
enables thus to keep, to direct according to positioning 
instructions and, accessorily, to supply (recharge) the 
telephone 1 . 

20 The controlled positioning means comprise a control 

circuit of motors 103 acting upon two motors 104 and 105 by 
motor control signals CM. One of the motors referenced as 

105 is fixed immobilised by its stator on the second bracket 

106 and its rotor drives the stator of the other motor 
25 referenced as 104 whereof the rotor is fixed to the first 

bracket 100. The line V and the link OM of the connector 101 
reach the second bracket 106 through the first bracket 100. 
The second bracket 106 includes a charger 102 (to recharge 
the telephone 1) receiving an external power supply, in 
30 particular mains from the public electric distribution network, 
and which enables, moreover, power supply of the controlled 
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positioning means (control circuit 103 and motors 104 and 
105). The positioning instructions issued from the control 
member 8 of the telephone 1 and acting upon the control 
circuit 103 are carried by the link OM. The initial source of the 
5 positioning instructions is preferably the analysis and 
diagnostic module which uses the link D towards the control 
member (for example following of a person or a face). It is 
however possible to transmit over the telephone, from of the 
telephone network, such positioning instructions which are 

10 then received by the receiver 4 of the telephone 1 and sent by 
the link OM by dint of the control member 8. In the latter case, 
the orientation may for instance be obtained from another 
telephone while depressing its keys or, if using computer 
means, in particular INTERNET, if moving one/several display 

15 cursors (mouse) of a computer terminal on the telephone 
network with computer gateway. 

The line V and the link OM are preferably materialised, 
in particular by a wire link, possibly by or associating one or 
several rotary contacts (the first and the second bracket being 

20 mobile relative to one another). In a more enhanced version, 
the line V and the link OM are de-materialised, for instance an 
electromagnetic induction means enabling to recharge the 
power supply unit 10 of the telephone and the link OM taking 
place over radio waves (for example BLUETOOTH®), the 

25 second bracket comprising then moreover compatible 
radiocommunications means connected to the motor control 
circuit 103. 

Preferably, the rotors of the motors are shifted by 90° to 
enable efficient scanning of the shooting space. Preferably 
30 there will a maximum angular bottoming, i.e. 360°, for the 
plane in space where the scene picture elements are most 
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likely to be found. For instance for a telephone arranged on a 
table in a room, it is preferable to scan, for horizontally 
shooting, the whole room (360°) where a person is more likely 
to be found than on the ceiling. On the other hand, due to 
5 mechanical constraints, in particular the presence of the 
brackets, there are zones where shooting is impossible. 

It goes without saying that the possibilities of angular 
space scanning from the viewing point are adapted relative to 
the needs and the implementation cost. Thus, if two motors 

10 have been represented on Figure 6, which enables volume 
space scanning, in a simplified version, a single motor may be 
implemented, preferably, enabling scanning in a plane where 
the picture elements are most likely to be found. 

The motors are preferably stepping motors. An indexing 

15 means (for locating the origin position) may be implemented 
either directly on the rotors, or, in a more complex fashion, by 
analysing pictures. The indexing means enables to know a 
reference position from which it is possible to determine the 
displacement (number of steps for stepping motors, which 

20 corresponds to an angular displacement which can be 
calculated). The indexing means, in a variation, may send a 
signal back to the telephone 1 over the bi-directional link OM. 
The positioning interface may also include other sensors such 
as for instance positioning sensors, in particular angular, 

25 speed and/or acceleration, battery level or power supply 
sensors... whereof the information is sent back to the 
telephone. 

Figure 7 enables to visualise the arrangement of the 
telephone 1 of the invention in the first bracket 100, itself in 
30 the second bracket 106. In the resting position, the telephone 
1 is substantially vertical and the shooting means of the 
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telephone has a shooting axis substantially horizontal. Thanks 
to the motor 105, the first bracket 100 may rotate with respect 
to the second bracket 106 enabling the shooting axis to scan 
space in substantially horizontal plane. The motor 104, for its 
5 own part, enables to toggle the first bracket 100 with respect 
to the second bracket 106, i.e. to enable the telephone 1 
leaving its substantially vertical position for positions close to 
the horizontal, whereas the shooting axis moves up and down 
and conversely. 

10 As indicated for Figures 6 and 7, it has been assumed 

that the second bracket is arranged at a fixed location in 
space. However, in other variations of the invention, the 
telephone 1 may be made mobile in space by using 
one/several brackets which possess means to move in said 

15 space. In such cases, it appears preferable that the external 
power supply is not connected permanently, unless it has 
reduced displacement radius (the length of the external power 
supply cord) and a stand-alone power supply means, in 
particular of rechargeable battery type, is implemented in the 

20 mobile bracket of the telephone. In such a version, the 
simplest version with rolling means, the device of Figures 7 
and 8 includes moreover on the internal face of the second 
bracket taxiing and directional means controlled by dint of the 
link OM. In another variation, the telephone 1 may be 

25 arranged substantially vertically on a supporting cart 
comprising driving and directional wheels (an additional 
controlled toggling means of the telephone which may be 
implemented). It appears that the controlled means of 
displacement on the ground (possibly another referential one) 

30 are adapted to the uses contemplated, as well as simple 
rolling means (wheels, caterpillar) or articulated legs, suction 
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devices... Similarly if displacement in a three-dimensional 
space is requested, the displacement means will be suited to 
said space. Conventional robotics management tools may be 
usefully implemented in such applications. 
5 As indicated previously and thanks to the possibility of 

hardware and functional configuration of the modules STN of 
the GVPP in relation to an application program by dint of the 
API, it is also possible to use the perception processor as a 
video picture compression system as described in 

10 FR98/16679 and WO99/00425 in order to improve the quality 
and the fluidity of the pictures transmitted over the telephone 
1, 110, whereas a higher volume of compressed information 
may be transmitted over the transmission channel. Thus, after 
a diagnostic leading to a positive decision causing a 

15 connection to the network, the perception processor may be 
reconfigured to operate in video compression. It may be 
understood that in such a case, the receiving means 130 
and/or the server network in case when INTERNET is used as 
a link with the remote apparatus 130, compatible decoding 

20 means will be placed therein. In such a case, a connection 
between two telephones of the invention is particularly 
interesting since they both include processor perception 
means which are compatible and configurable for 
compression/decompression. 

25 It is obvious that the invention may be applied in divers 

fashions and in particular as regards the diagnostics based on 
the analyses which may produce actions of various types: 
simple alarms, transmission of computer messages, 
transmission of video and/or sound data, possibly multimedia 

30 to various pieces of equipment of telephone, possibly 
computer, type, by dint of a radio-telephone network to which 
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the mobile telephone of the invention is connected. The 
applications of the invention are also multiple, for private or 
family surveillance with a conventional video mobile telephone 
comprising the analysis and diagnostic module as well as for 
5 professional applications and in this case, the telephone of 
the invention may be a specific realisation, in particular 
ruggedized mobile telephone or in the form of an industrial 
module. These applications may in particular derive from 
inventions implementing space-time-division neurons and 
10 described in the applications whereof the inventor is Monsieur 
PIRIM and which are in particular listed in the first portion of 
the description. 



